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earned patent term adjustment. See 37 CFR 1 .704(b). 
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2a)IEl This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) D Claim(s) is/are pending in the application. 
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DETAILED ACTION 

Response to Arguments 

1 . Applicants arguments filed 8/5/2004 have been fully considered. The Applicant 
in correct in stating that Trubko cannot be used as prior art therefore the Examiner 
widthdraws the rejection of claims 30-44 and issue an new final rejection based on art 
that does indeed predate the current invention included below. 
2. 

3. The Applicant argues that Nayar does not teach using a reflector that satisfies a 
single viewpoint constraint. The Examiner disagrees with this assertion. A careful 
reading of the reference especially column 9 lines 49-62 clearly shows that his reflector 
allows for reflection of a hemispherical scene from a single viewpoint. The Examiner 
further disagrees with the applicant in his assertion that since Nayar calls his curve 
substantially parabolic it is impossible for it to read on the claimed constraint. However, 
Nayar's shape is not a true parabola, his equation in column 8 is not a parabolic 
equation, thus the shape that satisfies the equation cannot be a parabola either in the 
true definition of a parabola regardless of what he call it. Never the less, he does 
indeed teach that his shape whatever it is classified as, provides a view of a 
hemispherical scene from a single view point because he says it does on line 53 of 
column 9. 

4. As for the argument that Nayar does not teach the use of a controller for creating 
a mapping matrix that defines a relationship between the image plane and the 
perspective viewing window, the Examiner disagrees. Once translated into ordinary 



Application/Control Number: 09/821,648 
Art Unit: 2613 



Page 3 



English, this limitation simply says that there is a conversion that takes place from the 
coordinate system used by the Mirror, i.e. cylindrical, and the image plane or monitor, 
i.e. Cartesian. As noted in the first office action Nayar teaches this and his controller is 
his computer. 

Finally, the Examiner reminds the Applicant that while the claims are given their 
broadest reasonable interpretation, and while they are read in light of the specification, 
limitation from the specification, however, are not read into the claims. 

Therefore the Examiner stands behind his original rejection as it pertains to claims 1-29. 
Claims 30-44 are further rejected as well. 

Claim Rejections - 35 USC § 102 

1 . * The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

2. Claims 1-6 and 14-23 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Nayar. 

Claim 1, 14, 15 A method for generating a selectable perspective view of a portion of a 
hemispherical image scene, comprising the steps of: 
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acquiring an omnidirectional image on an image plane using a reflective mirror that 
satisfies a singl ( e viewpoint constraint and an image sensor; (See Nayar column 9 lines 
49-53) 

defining a perspective viewing window based on configuration parameters; (See column 
7 lines 62-65 and column 10 lines 56-65) 

defining a predetermined geometric relationship between the reflective mirror and the 
image plane; and (See column 9 line 63 through column 10 line 19) 
mapping each pixel in the perspective window with a corresponding pixel value in the 
omnidirectional image on the image plane using the configuration parameters. (See 
column 10 line 55 through column 11 line 55 as well as column 12 lines 25-35.) 

Claim 2, 18. The method of claim 1, wherein the configuration parameters defined in the defining 
step include at least one of a zoom distance defined as the distance from the focal point 
of said reflective mirror to said window, a pan angle defined as the angle between the x 
axis and a line through the focal point of said reflective mirror perpendicular to the x-y 
plane and a tilt angle defined as the angle between the x-y plane and a vector normal to 
said window. (See column 10 lines 55-65) 



Claim 3, 19. The method of claim 2, wherein the defining step is conducted via a user interface 
through which a user enters data corresponding to at least one of a desired zoom 
distance, pan angle, or tilt angle. (See Column 7 lines 62-65) 

Claim 4, 16, 17, 20 The method of claim 1, wherein the mapping step includes the step of 
generating a mapping matrix by: 

applying a ray tracing algorithm to each pixel in the perspective viewing window to 
determine a corresponding reflection point on the reflective mirror; and projecting each 
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reflection point to a focal point of the image sensor to determine the corresponding 
location in the omni directional image on the image plane. (See figures 4, 5, and 6) 

Claim 5, 22, 23. The method of claim 4, further comprising the step of storing the mapping matrix 
in a module having a memory. (See Nayar figure 1 A item 125. Computers all have 
memory) 

Claim 6, 21. The method of claim 1 wherein the step of defining a perspective viewing window 

defines the perspective viewing window as a panoramic viewing window. See column 1 1 
lines 25-30) 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 7-9, 24 and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nayar as applied to claim 1 above, and further in view of Chahl and 
Mick. 

5. 

Claim 7, 24: The method of claim 1, further comprising the steps of: calculating a residual image 
based on a difference between a reference omni-directional image and a sequential 
omni-directional image; determining if the residual image contains any value that exceeds 
a predetermined threshold; and classifying any value that exceeds the predetermined 
threshold as an anomaly. (Nayar clearly teaches that his system can be used for 
surveillance purposes see column 1 lines 25-30. While he is not specific as to his motion 
detection means Chahl clearly teaches that motion detection can be done in a panoramic 
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surveillance system see column 6 lines 25-32, However, he does not specify how he 
does his motion detection. However, at the time the invention was made, it was well 
known in the art of surveillance, that in order to detect video motion, the conventional way 
was to compare subsequent video images and if a large enough difference in the images 
is detected, it is considered motion. As proof of the Examiners statements he includes 
Mick, see abstract. Therefore, at the time the invention was made, it would have been 
obvious to one of ordinary skill in the art, to include motion detection capabilities in 
Nayars system in order to adapt it for use in the surveillance application he says it can be 
used for.) 

Claim 8: The method of claim 7, further comprising the steps of: calculating the configuration 
parameters for the perspective viewing window from the anomaly; and selectively 
focusing the perspective viewing window on the anomaly using the calculated 
configuration parameters. (See Nayar column 10 lines 32-64) 

Claim 9, 25. The method of claim 7, further comprising the step of activating an alarm if at least a 
portion of the residual image exceeds a predetermined threshold. (See Chahl abstract.) 

6. Claim 10-13,26-29 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nayar as applied to claim 1 above, and further in view of Baker. 

Claim 10, 26. The method of claim 1, further comprising the steps of: detecting a location of a 
sound source in the image scene; and adjusting the perspective viewing window based 
on the detected location of the sound source. (While Nayar does not include this feature, 
he does state in column 1 line 26 that his system could be used for teleconferencing. 
Baker on the other hand, discloses a teleconferencing imaging system that includes a 
panoramic imaging means including an audio detection circuit that can locate the source 
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of a sound and image it. See Baker abstract. Therefore, at the time the invention was 
made it was well known in the art, that imaging the source of a sound was desired, thus it 
would have been obvious to one of ordinary skill in the art, to include the audio detection 
system disclosed by Baker into Nayar's imaging apparatus to make the invention as 
claimed. 

Claim 11, 12, 27, 28. The method of claim 1, further comprising the step of transmitting the omni- 
directional image via the Internet. (At the time the invention was made, use of the 
internet for transmitting images was well known in the art. Once the image is processed 
as in the case of the current invention, data is data, therefore it doesn't matter that the 
image came from an omni-directional imaging means. Never the less, while Nayar is 
silent on use of the Internet, Baker teaches that teleconferencing involves transmitting 
both the audio and video data to a remote site for viewing. See column 1 lines 16-17. 
Since teleconferencing implies use of the phone system, and access to the internet is 
achieved via the phone network, it would have been obvious to one of ordinary skill in the 
art to transmit the image via a server or any computer capable of handling the job in 
order to provide a suitable teleconferencing system.) 

Claim 13, 29. The method of claim 1 , further comprising the step of forming a two way 

transmission link between the image sensor and a remote display, wherein the two-way 
transmission link transmits at least one of the omni-directional image, the perspective 
viewing window, and an audio signal. (Teleconferencing is a two way transmission link.) 

7. Claim 30 rejected under 35 U.S.C. 103(a) as being unpatentable over Nayar as 
applied to claim 1 above, and further in view of Korein. 
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30. (new) The improved imaging apparatus of claim 14, wherein said reflective mirror is a hyperbolic 
mirror having a hyperbolic cross-section. (While it is unclear as to what shape Nayar's reflector is, i.e. the 
equation that defines it is not that of a parabola nor a sphere, it does resemble that of a hyperbola and it 
satisfies the single viewpoint limitation see Nayar column 9 lines 49-53, never the less at the time the 
invention was made, as admitted by the applicant in his own specification, it was well known in the art that 
a hyperbola could be used to view a panoramic scene from a single viewpoint, and has already been in 
use for such an application, see Korein column 9 line 59 wherein he teaches the use of a hyperbolic 
mirror. Therefore at the time the invention was made, it would have been obvious to one of ordinary skill 
in the art to use a hyperbolic mirror in Nayar's invention since the single viewpoint constraint is satisfied 
using a hyperbolic mirror.) 

8. Claims 31-38 and 44 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nayar in view of Korein. 

Claim 31, 44. (new) An imaging apparatus for generating a two-dimensional image, comprising: a 
reflective hyperbolic mirror having a hyperbolic cross-section; 
an image sensor optically coupled to said reflective mirror that generates two- 
dimensional image data signals based on an omni directional image reflected by said 
mirror; ; and a controller coupled to the image sensor, wherein the controller defines a 
perspective viewing window and includes a mapping matrix .generator that defines a 
geometric relationship between the image sensor and the perspective viewing window 
such that at least a portion of the omni directional image on the image plane can be 
mapped to the perspective viewing window. (See Claim 14 and 30 rejections) 

Claim 32. (new) The imaging apparatus of claim 31, wherein the reflective mirror creates a one- 
to-one correspondence between pixels in the omnidirectional image and pixels in the 
perspective viewing window. (See Nayar column 10 lines 20-31 .) 
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Claim 33. (new) The imaging apparatus of claim 31 , wherein the controller maps the omni 
.directional image to the perspective viewing window by mapping each pixel in the 
perspective viewing window with a corresponding pixel value in the omnidirectional 
image. (See Nayar Column 12 lines 25-34) 

Claim 34. (new) The imaging apparatus of claim 14, wherein parameters defining the perspective 
viewing window include at least one of a zoom distance defined as the distance from the 
focal point of said reflective mirror to said window, a pan angle defined as the angle 
between the x axis and a line through the focal point of said reflective mirror 
perpendicular to the x-y plane and a tilt angle defined as the angle between the x-y plane 
and a vector normal to the perspective viewing window. (The Examiner is assuming that 
this claim was supposed to be dependent from claim 31 not 14. In either case see (See 
Nayar column 10 lines 55-65) 

Claim 35. (new) The imaging apparatus of claim 34, further comprising a user interface through 
which a user enters data corresponding to at least one of a desired zoom distance, pan 
angle, or tilt angle. (See Nayar Column 7 lines 62-65) 

Claim 36. (new) The imaging apparatus of claim 31, wherein the controller generates a mapping 
matrix by applying a ray tracing algorithm to each pixel in the perspective viewing window 
to determine a corresponding reflection point on the reflective mirror and then projecting 
each reflection point to a focal point of the image sensor to determine the corresponding 
location on the omni directional image. (See Nayar figures 4, 5, and 6) 

Claim 37. (new) The imaging apparatus of claim 31, wherein the perspective viewing window is a 
panoramic viewing window. (See Nayar column 1 1 lines 25-30) 
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Claim 38. (new) The imaging apparatus of claim 31 , further comprising a memory for storing a 

mapping matrix generated by said mapping matrix generator. (See Nayar figure 1A item 
125. Computers all have memory) 

9. Claims 39 and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nayar and Korein as applied to claim 31 above, and further in view of Chahl and 
Mick. 

Claim 39. (new) The imaging apparatus of claim 31, wherein the controller calculates a residual 
image based on a difference between a reference omnidirectional image and a 
sequential omnidirectional image to detect an anomaly and uses the anomaly to calculate 
parameters for the perspective viewing window so that the perspective viewing window 
focuses on the anomaly. (Nayar clearly teaches that his system can be used for 
surveillance purposes see column 1 lines 25-30. While he is not specific as to his motion 
detection means Chahl clearly teaches that motion detection can be done in a panoramic 
surveillance system see column 6 lines 25-32. However, he does not specify how he 
does his motion detection. However, at the time the invention was made, it was well 
known in the art of surveillance, that in order to detect video motion, the conventional way 
was to compare subsequent video images and if a large enough difference in the images 
is detected, it is considered motion. As proof of the Examiners statements he includes 
Mick, see abstract. Therefore, at the time the invention was made, it would have been 
obvious to one of ordinary skill in the art, to include motion detection capabilities in 
Nayars system in order to adapt it for use in the surveillance application he says it can be 
used for.) 
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Claim 40. (new) The imaging apparatus of claim 39, further comprising an alarm that is activated 
if at least a portion of the residual image exceeds a predetermined threshold. (See Chahl 
abstract.) 

10. Claim 41-43 rejected under 35 U.S.C, 103(a) as being unpatentable over Nayar 
and Korein as applied to claim 31 above, and further in view of Baker. 

Claim 41 . (new) The imaging apparatus of claim 31 , further comprising an acoustic sensor 
coupled to the controller for detecting a sound source within a scene of said 
omnidirectional image reflected by said mirror, wherein the controller adjusts the 
perspective viewing window based on a location of the sound source. (While Nayar does 
not include this feature, he does state in column 1 line 26 that his system could be used 
for teleconferencing. Baker on the other hand, discloses a teleconferencing imaging 
system that includes a panoramic imaging means including an audio detection circuit that 
can locate the source of a sound and image it. See Baker abstract. Therefore, at the 
time the invention was made it was well known in the art, that imaging the source of a 
sound was desired, thus it would have been obvious to one of ordinary skill in the art, to 
include the audio detection system disclosed by Baker into Nayar's imaging apparatus to 
make the invention as claimed.) 



Claim 42. (new) The imaging apparatus of claim 31, further comprising an image transmission 
system for transmitting image output by said image sensor via the Internet. (At the time 
the invention was made, use of the internet for transmitting images was well known in the 
art. Once the image is processed as in the case of the current invention, data is data, 
therefore it doesn't matter that the image came from an omni-directional imaging means. 
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Never the less, while Nayar is silent on use of the Internet, Baker teaches that 
teleconferencing involves transmitting both the audio and video data to a remote site for 
viewing. See column 1 lines 16-17. Since teleconferencing implies use of the phone 
system, and access to the internet is achieved via the phone network, it would have been 
obvious to one of ordinary skill in the art to transmit the image via a server or any 
computer capable of handling the job in order to provide a suitable teleconferencing 
system.) 



Claim 43. (new) The imaging apparatus of claim 31 , further comprising: a remote display coupled 
to the image sensor; a first speaker and first microphone coupled to the image sensor; 
and a second speaker and second microphone coupled to the remote display, wherein 
the first and second speakers and first and second microphones form a two-way 
transmission link between the image sensor and the remote display. (Teleconferencing 
is a two way transmission link.) 

Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
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shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Charles E Parsons whose telephone number is 703- 
305-3862. The examiner can normally be reached on M-TH 7AM to 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris Kelley can be reached on 703-305-4856. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private. PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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